Confocal imaging is a well-known imaging technique in which light scattered from a target is spatially filtered before it is detected. Spatial filtering can reduce image artifacts such as unwanted scattering and reflections from either the target or components within the imaging device, and can provide improved image contrast, as well as isolate features of interest. Confocal imaging devices have been designed and implemented for a wide variety of microscopy, dark-field, fluorescence, polarization sensitive, nonlinear optical, interferometric, and ophthalmic imaging applications.
In a laser scanning confocal imaging system, the illumination light is focused to a point or line and scanned across the target to obtain an image of the entire field of view. Light scattered from the target is typically descanned using the same scanning element and directed through an aperture to a photosensitive detector. By synchronizing the scanning with the exposure timing of the detector, a two-dimensional image of the target can be constructed. The insertion of an aperture at a plane conjugate to the target restricts the amount of out-of-focus light that can reach the detector. Laser scanning confocal imaging systems have been adapted for retinal imaging in scanning laser ophthalmoscopes, and for three-dimensional interferometric imaging in optical coherence tomography devices.
U.S. Pat. No. 7,831,106 proposes the use of a laser scanning confocal imaging system without descanning. In this design, a confocal aperture is created by an electronic rolling shutter detection means on a two-dimensional pixel array sensor. During each frame, the rolling shutter progressively scans in one-dimension across the active sensor region with a shutter width related to the total frame exposure time. At each shutter position, the pixel values along the length of the shutter are obtained by integrating the charge accumulated across only the width of the shutter. Light incident on the sensor outside the rolling shutter width is not captured.
The electronic rolling shutter is a fundamentally different form of detection than a global shutter, which integrates charge across the entire active area during the exposure time. Charge coupled device (CCD) sensors are an example of a global shutter sensor; many complementary metal-oxide semiconductor (CMOS) sensors use rolling shutter technology, though they are now also available with global shutters. In non-confocal imaging applications, rolling shutter sensors are commonly used in the place of a global shutter sensor due to the low cost of CMOS technology. In these cases, a rolling shutter is generally considered a detriment to imaging performance since target motion in the same or opposite direction of the rolling shutter will appear distorted. Several research groups have investigated post-processing techniques to reduce motion blur and other distortions in non-confocal cameras that use rolling shutter sensors.
The confocal imaging system design proposed by U.S. Pat. No. 7,831,106 scans an illumination line across the target in synchrony with the rolling shutter detection of a sensor placed at a conjugate target plane. This approach allows the aperture width to be adjusted electronically in real-time, and permits an operator to adjust the trade-off between the amounts of light detected and confocal spatial filtering for a given target. Furthermore, the relative timing between the shutter position and scanning angle can be adjusted electronically in real-time in order to perform dark-field imaging. When the frequency or polarization components of the light are spatially separated, the rolling shutter can also be used to filter the frequency or polarization of the light scattered from the target.
The use of the electronic rolling shutter as a confocal aperture permits adjustments to the aperture position and width in pixel increments. Compared to mechanical apertures, the use of the rolling shutter as a confocal aperture is a cost-effective approach that permits rapid, quantifiable, accurate and reliable adjustments. However, U.S. Pat. No. 7,831,106 requires the formation of a slit that is scanned across the field of view of the target. With a single scanner, the simultaneous illumination of a target with a second spatially offset slit, or other more complex patterns, requires additional illumination pathways to the scanner. These illumination pathways are difficult to align with high precision and each enable only a few additional illumination geometry configurations. A confocal imaging method and device that further provides flexible and precise electronically-controlled real-time adjustments to the illumination geometry using a common illumination pathway and that is compact, robust, reliable, and cost-effective would be appreciated.
In a programmable array microscope (PAM) confocal imaging system, the illumination pattern incident on the target is adjustable using a spatial light modulator, such as a digital micromirror array or liquid crystal display. In this configuration, a confocal image is constructed from multiple frames that are acquired while the target is illuminated with a series of alternating dot lattice or pseudo-random patterns. The light returning from the target is spatially filtered by the spatial light modulator and global shutter CCD; only the sensor pixels conjugate to the “on” pixels of the micromirror array are used to construct the final image. A laser contrast speckle imaging system with a spatial light modulator and CMOS sensor has been reported that measures blood flow changes. This system uses a spatial light modulator frame rate that is many times slower than that of the sensor. This prevents the spatial light modulator from rapidly projecting a sequence of narrow illumination lines that could continuously overlap with the rolling shutter during a single frame. Furthermore, as reported, this system does not use the rolling shutter of the CMOS sensor as a confocal aperture; the CMOS sensor could be substituted for a global shutter CCD and achieve substantially the same performance.
PAM systems for fluorescent microscopy have been implemented using two pixel array detectors to collect both the in-focus and out-of-focus light that scatters from a target. In these systems, the “off” and “on” angular orientation of the micromirror array elements is used to direct the light scattered from the target to each of the two detectors. A fluorescence-based PAM has also been implemented in which the scattered light is spatially filtered with a fiber optic cable and measured using a spectrometer for hyperspectral and fluorescence lifetime imaging.
The ability of spatial light modulators to rapidly change the modulation pattern used to illuminate an object makes them well-suited for structured light illumination applications, such as phase measuring profilometry and fringe-projection microscopy, in which a series of images taken with periodic illumination fringes can be used to perform spatial filtering. In these systems, confocal imaging is achieved with a global shutter CCD sensor. The use of a spatial light modulator is particularly attractive due to its ability to change the frequency and shape of the structured light illumination in real time. Although spatial light modulators are unable to continuously scan a beam of light across a sample, they can simulate the effect by rapidly projecting a series of modulation patterns.
The use of a digital micromirror array in a PAM system has attracted interest due to its low cost, illumination pattern flexibility, high mirror speed, and ever-increasing pixel resolution. However, the use of a series of illumination patterns to construct a confocal image requires the acquisition of multiple frames, during which time the imaging system is highly sensitive to motion artifacts. U.S. Pat. No. 5,923,466 addresses this difficulty by proposing a dual-pass system, in which the light returning from the target is directed back through the spatial light modulator prior to being detected. This approach is similar to the laser scanning designs discussed above. While dual-pass designs have been proven effective for confocal imaging, they typically require a beam separating element between the scanning component and the source to direct the light return from the target to the detector. In a confocal dual-pass spatial light modulator system, the addition of a beam separating element and optical design of the associated detection pathway prevents the use of a fully integrated illumination source and micromirror array, as provided in digital light projectors. Therefore, a method and device for confocal imaging that enables the use of cost effective and robust spatial light modulators that are integrated with the illumination source, such as currently available compact and lightweight digital light projectors, which can be handheld, would be appreciated. A method and device for confocal imaging that further removes the need to construct a confocal image using multiple sensor frames, as in fringe-projection microscopy, would also be appreciated.
U.S. Pat. Nos. 5,867,251 and 7,755,832 propose the implementation of a second spatial light modulator, driven in tandem with the first, to act as a second aperture and to restrict the light returning from the target that reaches the detector. This approach results in a complex system that requires precise alignment and timing control of the spatial light modulators; such a system has not, to the present inventors' knowledge, been reduced to practice and reported in the literature. A method and device that provides flexible, cost-effective, and robust aperture control in a PAM system would be appreciated.
Optical coherence tomography (OCT) based systems perform imaging by analyzing the interference between broadband light returning from a target and light reflected in a reference arm with a known path delay. The most common implementation uses a Michelson interferometer, and determines the backscattered intensity with respect to sample depth at one transverse point on the sample at a time. A three-dimensional image is built up by raster scanning the beam across the sample. Numerous comprehensive reviews of the progress and development of OCT-based systems and their applications can be found in the literature.
OCT systems broadly belong to two classes: time-domain OCT and frequency-domain OCT. The frequency-domain OCT class is further separated into spectral-domain OCT (SD-OCT) and swept source OCT (SS-OCT) design architectures. Spectral-domain and swept source OCT are commonly also referred to as Fourier-domain OCT and optical frequency domain imaging, respectively, by those skilled in the art.
In the case of time-domain OCT, the reference path delay is commonly mechanically stepped across the full sample depth range. At each reference arm position, the intensity of the interference is recorded by a photodetector, yielding the scattering depth profile for the reference arm range of motion. The speed at which the reference path delay can be mechanically scanned typically limits the acquisition rate. Although time-domain OCT can rapidly provide en face images of the target at a single depth position, time-domain OCT suffers from poorer sensitivity as compared to the class of frequency-domain OCT systems.
In SS-OCT systems, the illumination source is a tunable laser with a narrow instantaneous bandwidth. An axial scan is constructed by sweeping the frequency of the laser through its gain bandwidth while measuring the interference signal intensity.
In SD-OCT systems, the reference arm remains fixed, and the light from the reference and sample arms is measured with a spectrometer, commonly comprised of a fixed grating and a line-scan sensor. An inverse Fourier transform is applied in post-processing to reconstruct the scattering depth profile, achieving the same axial resolution as obtained in time-domain OCT systems. The electronic line-scan sensor provides a faster scan rate than the mechanical mirror scan rates achieved in TD-OCT. In addition, by spreading the imaging spectrum across many pixels, the noise is reduced, permitting a higher sensitivity with respect to TD-OCT. However, the use of a single sensor in SD-OCT makes these systems more susceptible to random intensity noise than swept-source and time-domain OCT systems that use a pair of photodetectors for balanced detection.
Each of the above OCT systems typically use a broad bandwidth source to achieve a high depth resolution and a pair of galvanometer scanners to quickly raster scan a spot over the target. Spatial light modulators, such as digital micromirror arrays, have been used in catheter-based OCT applications where their small size is an advantage. In these systems, the light returning from the target is descanned prior to detection.
The amount of reference arm light used in an OCT system is typically adjusted using a partial reflector or other variable attenuator depending on the target being imaged. To maximize the dynamic range of the system, the reference arm power is increased until the light detected at all points on the target is just below saturation. When there are differences in the intensity of light return across the field of view of the target, the noise floor of the imaging system can be limited by the dynamic range of the sensor's analog to digital converter, rather than the sensitivity limit caused by either the shot or dark noise.
Line-scanning parallel SD-OCT systems have been reported that use a two-dimensional detector and illuminate the target with a line. In these systems, the light returning from the target is descanned and sent through a linear aperture to reject light outside the illumination focal volume. During detection, one axis of the two dimensional sensor represents the frequency of the interferogram, while the other is the lateral position along the target. An advantage of parallel SD-OCT systems is the simultaneous acquisition of depth and lateral scans during each frame exposure, permitting an increase in imaging speed.
A method and device for OCT imaging that enables the use of an illumination source integrated with a spatial light modulator, combined with a confocal rolling shutter means of detection, would be appreciated. Specifically, the use of an integrated source and spatial light modulator, such as a compact and lightweight digital light projector, which can be handheld, would be more compact and cost-effective than existing OCT designs. The modification of the illumination modulation pattern to reduce differences in the intensity of light return across the field of view of the target, and thereby increase the dynamic range of the image, would be appreciated.
Adaptive optics (AO) imaging systems strive to correct aberrations in the detected light to produce higher resolution and higher contrast images. AO has been extensively used in biomedical imaging, microscopy and for imaging structures in the ocular fundus in animals and humans.
AO imaging systems typically measure and correct for wavefront aberrations in the light returning from the target. Wavefront aberrations are measured using a wavefront detector in a conjugate Fourier plane, such as a Shack-Hartmann sensor, which consists of a lenslet array and global shutter 2-dimensional pixel array detector. Feedback from the Shack-Hartmann wavefront detector is used to drive one or more wavefront controllers, typically deformable mirrors or liquid crystal spatial light modulators. A scanning laser opthalmoscope AO system using a woofer-tweeter set of deformable mirrors for coarse and fine wavefront adjustments over a wide field of view has been demonstrated with retinal tracking.
To minimize unwanted aberrations, ophthalmic AO systems typically require a large amount of space, with separate conjugate pupil planes required for the horizontal scanner, vertical scanner, wavefront detector, and wavefront controller. To the present inventor's knowledge, all ophthalmic scanning laser AO systems have performed de-scanning prior to detecting and modifying the wavefront. A more compact AO imaging device that uses cost effective and robust spatial light modulators that are integrated with the illumination source, such as currently available compact and lightweight digital light projectors, which can be handheld, would be appreciated. A method and device for AO imaging that could illuminate selected target locations suitable for obtaining accurate wavefront detection measurements, correct optical aberrations, and image those target locations that would otherwise return poorer image contrast, image focus, or less spatially filtered light, would be appreciated. A method and device for AO imaging that uses the spatial filtering properties of a cost-effective rolling shutter sensor for detecting changes in the wavefront would be appreciated.
Optical synthetic aperture imaging systems take multiple measurements of a target from varied detection angles and interfere the returning wavefront with a reference beam to gain amplitude and phase information of the light returning from the target. By introducing the reference beam at a large off-axis angle with respect to the angle of light returning from the target, the spatial frequency content is downshifted, permitting enhanced image resolution, depth of focus, field of view and working distance. A similar approach, with detection in the Fourier plane of the target, has reported relaxed constraints on the attainable field of view for a given number of pixels in the detector array. Another reported approach records a series of holograms after spatially filtering the light returning from the target in the Fourier plane.
One application of synthetic aperture imaging has been for use in microscopy, where the technique enables high resolution imaging with lower numerical aperture optics. A device and method to illuminate a target with a specific range or set of angles under real-time software control would be appreciated. A device and method that spatially filters the light returning from the target according to a specific angle, and allows flexible, reproducible and accurate setting and control of the angle of detection with respect to the angle of illumination would be appreciated. A device and method that performs synthetic aperture retinal imaging as a more cost-effective and compact alternative to adaptive optics imaging would be appreciated.
The present invention is intended to improve upon and resolve some of these known deficiencies within the relevant art.